Attorney Docket No. 5034-0001 
Customer Number 28777 

AMENDMENTS TO THE CLAIMS 

This listing of claims replaces all prior versions and listings of claims in the application: 
Listing of Claims 

1. (Currently Amended) A coherent optical detection system receiving an 
incoming optical signal in an optical communications network, said system comprising: 
a local oscillator emitting light; 

a phase diverse hybrid for generating two replicas of the incoming signal and two 
replicas of the local oscillator light, said phase diverse hybrid combining the first replica 
of the incoming optical signal and the first replica of the local oscillator light into a first 
output and combining the second replica of the incoming optical signal and the second 
replica of the local oscillator light into a second output and wherein said local oscillator 
does not have to be phase locked to the incoming optical signal; 

wherein the phase relationship between the optical signal and the local oscillator 
light in the first output is different from 0 degrees and different from 180 degrees 
compared to the phase relationship between the local oscillator light and the optical 
signal in the second output and the state of polarization of the optical signal relative to 
the local oscillator light in the first output is not orthogonal to the state of polarization of 
the optical signal relative to the local oscillator light in the second output; and 

two photodetectors communicating with the phase diverse hybrid, wherein said 
two photodetectors receive optical signals from the two outputs and convert them to 
electrical signals; 

the electrical signals received by the two photodetectors are digitized by two A/D 
converters to create digital values; and 

a digital signal processor multiplies the digital values from the two A/D converters 
by two coefficients and sums them to form a complex number, then multiplies the 
complex number by a rotating phase factor to provide a complex representation of the 
envelope of the electric field of the incoming optical signal or a component of the 
complex representation of the envelope of the electric field of the incoming optical 
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signal the rotating phase factor having a frequency and a phase equal to estimates of 
the frequency and the phase of the incoming optical signal with respect to the local 
oscillator light- 
wh e r e by th e ele ctr i ca l s i gna l s ar e process e d to prov i d e a comp le x r e pr e s e ntat i on 
of th e e nv el op e of th e ele ctr i c f iel d of th e i ncom i ng opt i ca l s i gna l or a compon e nt of th e 
comp l ox roprosontat i on of tho onvo l opo of tho o l octr i c f i o l d of tho i ncom i ng opt i ca l 

2. (Canceled) 

3. (Withdrawn) The coherent optical detection system of claim 1 wherein: 
the electrical signals from the two photodetectors are separately multiplied by two 

electrical waveforms, each waveform being periodic and having the same frequency as 
the difference frequency between the local oscillator light and the incoming optical 
signal; 

the waveforms having different phases from one another; and 

the electrical signals corresponding to the results of two multiplication operations 

are summed to give an estimate of a component of the complex representation of the 

incoming optical signal. 

4. (Canceled) 

5. (Withdrawn) The coherent optical detection system of claim 4 wherein the 
digital signal processor compensates for the chromatic dispersion experienced by the 
incoming optical signal. 

6. (Withdrawn) A coherent optical detection system receiving an incoming 
optical signal in a fiber optics network, said system comprising: 

a local oscillator emitting light; 

an optical mixing hybrid for generating at least four replicas of the incoming 
optical signal and four replicas of the local oscillator light, said optical mixing hybrid 
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combining the replicas of the incoming optical signal and the replicas of the local 
oscillator light into at least four outputs and wherein said local oscillator does not have 
to be phase locked to the incoming optical signal; 

wherein four outputs of the at least four outputs of the optical mixing hybrid can 
be selected such that the Jones vector of the optical signal relative to the local oscillator 
light at each of the four selected outputs is distinct from the Jones vector of the optical 
signal relative to the local oscillator light at the other three of the selected outputs; 

four photodetectors communicating with the optical mixing hybrid, wherein said 
four photodetectors receive optical signals from the four outputs; 

four A/D converters to receive electrical signals from the four photodetectors, 
said four A/D converters digitizing the received electrical signals; and 

a digital signal processor for performing computations on digital values from the 
four A/D converters to obtain information carried by the incoming optical signal without 
limitation to the state of polarization of the incoming optical signal. 

7. (Withdrawn) The coherent optical detection system of claim 6 wherein: 
the state of polarization of the optical signal relative the local oscillator light at the 

first and second of the selected outputs is not orthogonal, and the state of polarization 
of the optical signal relative the local oscillator light at the third and fourth of the selected 
outputs is not orthogonal, and the state of polarization at the first and second outputs is 
close to orthogonal to the state of polarization at the third and fourth outputs; and 

the phase relationship between the optical signal and the local oscillator light at 
the first output is approximately 90 degrees compared to the phase relationship 
between the optical signal and the local oscillator light at the second output, and the 
phase relationship between the optical signal and the local oscillator light at the third 
output is approximately 90 degrees compared to the phase relationship between the 
optical signal and the local oscillator light at the fourth output. 

8. (Withdrawn) The coherent optical detection system of claim 6 wherein the 
digital signal processor obtains a Jones vector representation or equivalent 
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representation that includes the amplitude, phase and polarization of the incoming 
optical signal. 

9. (Withdrawn) The coherent optical detection system of claim 8 wherein: 
the incoming optical signal includes two optical channels being combined so that 

the first optical channel of the two optical channels has a state of polarization close to 
orthogonal to the second optical channel of the two optical channels; and 

the digital signal processor performs computations to obtain information carried 
by each of the two polarization multiplexed channels in the incoming optical signal 
independent of the other channel. 

10. (Withdrawn) The coherent optical detection system of claim 6 wherein the 
digital signal processor produces an output which is the result of a signal processing 
operation on a plurality of samples over time of the Jones vector of the incoming optical 
signal. 

11. (Withdrawn) The coherent optical detection system of claim 10 wherein 
the digital signal processor compensates for the chromatic dispersion experienced by 
the incoming optical signal. 

12. (Withdrawn) A coherent optical detection system receiving an incoming 
optical signal in an optical communications network, said system comprising: 

two local oscillators, each local oscillator emitting light; 

an optical mixing hybrid for generating at least four replicas of the incoming 
optical signal and two replicas of light from each of the local oscillators, said optical 
mixing hybrid combining the incoming optical signal and the local oscillator light into at 
least four outputs and wherein said local oscillators do not have to be phase locked to 
the incoming optical signal; 

a first output of the four outputs having a first replica of the optical signal and a 
first replica of light from the first local oscillator of the two local oscillators, wherein the 
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state of polarization of the optical signal and the light emitted from the first local 
oscillator have a defined relationship; 

a second output of the four outputs having a second replica of the optical signal 
and a second replica of light from the first local oscillator of the two local oscillators, 
wherein the phase relationship between the optical signal and the local oscillator light in 
the first output is different from 0 degrees and different from 180 degrees compared to 
the phase relationship between the local oscillator light and the optical signal in the 
second output; 

a third output of the four outputs having a third replica of the optical signal and a 
first replica of light from the second local oscillator of the two local oscillators, wherein 
the state of polarization of the light from the second local oscillator with respect to the 
optical signal in the third output is close to orthogonal compared to the state of 
polarization of light from the first local oscillator with respect to the optical signal in the 
first output; 

a fourth output of the four outputs having a fourth replica of the optical signal and 
a second replica of light from the second local oscillator of the two local oscillators, 
wherein the phase relationship between the optical signal and the local oscillator light in 
the third output is different from 0 degrees and different from 180 degrees compared to 
the phase relationship between the local oscillator light and the optical signal in the 
fourth output; 

at least four photodetectors communicating with the optical mixing hybrid, 
wherein said four photodetectors receive optical signals from the four outputs; 

at least four A/D converters to receive electrical signals from the four 
photodetectors, said four A/D converters digitizing the electrical signals; and 

a digital signal processor for performing computations on digital values from the 
four A/D converters to obtain information carried by the incoming optical signal. 

13. (Withdrawn) A coherent optical detection system receiving an incoming 
optical signal containing information in a fiber optics network, said system comprising: 
a local oscillator emitting light; 
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an optical mixing hybrid for combining the incoming optical signal and the local 
oscillator light into at least one output; 

a photodetector communicating with the optical mixing hybrid, wherein said 
photodetector receives an optical signal from the output and converts it to an electrical 
signal; 

an A/D converter to receive the electrical signal from said photodetector, said 
A/D converter digitizing the electrical signal; and 

a digital signal processor for performing computations on digital values from the 
A/D converter, the digital signal processor producing an output which is the result of a 
signal processing operation on a plurality of samples over time of the complex envelope 
of the electric field of the incoming optical signal, the output having the information 
contained in the incoming optical signal. 

14. (Withdrawn) The coherent optical detection system of claim 13 wherein 
the digital signal processor at least partially reverses the effect of propagation of the 
incoming optical signal through an optical fiber transmission system. 

15. (Withdrawn) The coherent optical detection system of claim 14 wherein 
the digital signal processor compensates for the chromatic dispersion of the optical fiber 
transmission system. 

16. (Withdrawn) The coherent optical detection system of claim 15 wherein 
the signal processing operation performed by the digital signal processor compensates 
for the chromatic dispersion experienced by the optical signal by applying to the 
complex envelope of the incoming optical signal a convolution with a specified 
mathematical function, the mathematical function being close to the impulse response 
of the transfer function corresponding to a chromatic dispersion equal in magnitude and 
opposite to the chromatic dispersion of the optical fiber transmission system. 

17. (Withdrawn) The coherent optical detection system of claim 14 wherein 
the signal processing operation performed by the digital signal processor at least 
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partially reverses the effect of self phase modulation imposed on the incoming optical 
signal. 

18. (Withdrawn) The coherent optical detection system of claim 13 wherein 
the signal processing operation performed by the digital signal processor at least 
partially reverses the effect of multipath interference imposed on the incoming optical 
signal. 

19. (Withdrawn) The coherent optical detection system of claim 13 wherein 
the signal processing operation performed by the digital signal processor includes 
performing an optical filtering function on the complex envelope of the electric field. 

20. (Withdrawn) The coherent optical detection system of claim 13 wherein 
the signal processing operation performed by the digital signal processor improves the 
quality of the incoming optical signal, the digital signal processor applying an algorithm 
which utilizes parameters that are adjusted to give different signal processing functions, 
and the values of those parameters are chosen for improving the quality of the 
recovered signal. 

21. (Withdrawn) The coherent optical detection system of claim 20 wherein 
the signal processing operation that improves the quality of the recovered signal is a 
feedforward equalization-decision feedback equalization function. 

22. (Withdrawn) The coherent detection system of claim 20 wherein the 
signal processing operation that improves the quality of the recovered signal is a 
maximum likelihood sequence estimation function. 

23. (Withdrawn) The coherent optical detection system of claim 13 wherein 
the digital signal processor produces an output which is the result of a signal processing 
operation on a plurality of samples over time of the Jones vector of the incoming optical 
signal. 
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24. (Withdrawn) The coherent optical detection system of claim 23 wherein 
the digital signal processor compensates for the chromatic dispersion experienced by 
the incoming optical signal. 

25. (Withdrawn) The coherent optical detection system of claim 23 wherein 
the signal processing operation performed by the digital signal processor at least 
partially reverses the effect of polarization mode dispersion imposed on the incoming 
optical signal. 

26. (Withdrawn) A coherent optical detection system receiving an incoming 
optical signal having a plurality of wave division multiplexed (WDM) channels, said 
system comprising: 

at least one local oscillator emitting light; 

an optical mixing hybrid for combining the incoming optical signal and the local 
oscillator light into at least one output; 

a photodetector communicating with the optical mixing hybrid, wherein said 
photodetector receives optical signals from the output; 

an A/D converter to receive electrical signals from said photodetector, said A/D 
converter digitizing the electrical signals; and 

a digital signal processor for performing computations on digital values from the 
A/D converter, the digital signal processor estimating information carried on a first of the 
plurality of WDM channels that takes into account the other WDM channels and 
subtracts crosstalk imposed on the first WDM channel by at least one of the remainder 
of the plurality of WDM channels. 

27. (Withdrawn) The coherent optical detection system of claim 26 wherein 
the crosstalk arises because the optical spectrum of the first WDM channels is partially 
overlapped over the optical spectrum of a second WDM channel of the incoming optical 
signal 
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28. (Withdrawn) The coherent optical detection system of claim 26 wherein 
the crosstalk is cross phase modulation imposed on the first WDM channel by a second 
WDM channel during passage through an optical fiber transmission system. 

29. (Withdrawn) The coherent optical detection system of claim 26 wherein 
the crosstalk is caused by four wave mixing occurring when the plurality of WDM 
channels generates a four wave mixing product which at least partially overlaps the 
optical spectrum of the WDM channel that experiences the crosstalk. 

30. (Currently Amended) A method of receiving an incoming optical signal in a 
coherent optical detection system, said method comprising the steps of: 

emitting light from a local oscillator, said local oscillator not requiring a phase lock 
with the incoming optical signal; 

generating two replicas of the incoming signal and two replicas of the local 
oscillator light by a phase diverse hybrid; 

combining, by the phase diverse hybrid, the first replica of the incoming optical 
signal and the first replica of the local oscillator light into a first output; 

combining the second replica of the incoming optical signal and the second 
replica of the local oscillator light into a second output, wherein the phase relationship 
between the optical signal and the local oscillator light in the first output is different from 
0 degrees and different from 180 degrees compared to the phase relationship between 
the local oscillator light and the optical signal in the second output and the state of 
polarization of the optical signal relative to the local oscillator light in the first output is 
not orthogonal to the state of polarization of the optical signal relative to the local 
oscillator light in the second output; 

receiving optical signals from the two outputs by two photodetectors in 
communication with the phase diverse hybrid; and- 

converting the optical signals from the two outputs into electrical signalsir-the 
o l octr i ca l s i gna l s bo i ng procossod to prov i de a comp l ex roprosontat i on of tho onvo l opo 
of th e ele ctr i c f iel d of th e i ncom i ng opt i ca l s i gna l or a compon e nt of th e comp le x 
roprosontat i on of tho onvo l opo of tho o l octr i c Ho l d of tho i ncom i ng opt i ca l s i gna l . 
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digitizing the electrical signals by two A/D converters to create digital values; 

multiplying digital values from the two A/D converters by coefficients and 
summing them to form a seguence of complex numbers; and 

multiplying the complex numbers by a rotating phase factor whose freguencv is 
egual to an estimate of the freguencv of the signal with respect to the local oscillator 
light and whose phase is egual to an estimate of the phase of the incoming optical 
signal with respect to the local oscillator light to provide a complex representation or 
component thereof of the electric field envelope of the incoming optical signal. 

31. (Canceled) 

32. (Withdrawn) The method of receiving an incoming optical signal of claim 
30 wherein the step of converting the optical signals includes the steps of: 

multiplying the electric signals separately by two electrical waveforms, each 
waveform being periodic and having the same frequency as the difference frequency 
between the local oscillator light and the incoming optical signal and wherein the two 
electrical waveforms have different phases from one another; and 

summing the electrical signals corresponding to the results of the two 
multiplication operations to give an estimate of a component of the complex 
representation of the incoming optical signal. 

33. (Withdrawn) A method of receiving an incoming optical signal in a 
coherent optical detection system, said method comprising the steps of: 

emitting light from a local oscillator, said local oscillator not requiring a phase lock 
with the incoming optical signal; 

generating at least four replicas of the incoming optical signal and four replicas of 
the local oscillator light by an optical mixing hybrid; 

combining, by the optical mixing hybrid, the replicas of the incoming optical signal 
and the replicas of the local oscillator light into at least four outputs, wherein four 
outputs of the at least four outputs of the optical mixing hybrid can be selected such that 
the Jones vector of the optical signal relative to the local oscillator light at each of the 
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four selected outputs is distinct from the Jones vector of the optical signal relative to the 
local oscillator light at the other three of the selected outputs; 

receiving optical signals from the four outputs by four photodetectors in 
communication with the optical mixing hybrid; 

digitizing electrical signals, by four AID converters, from the four photodetectors; 

and 

performing computations, by a digital signal processor, on digital values from the 
four AID converters to obtain information carried by the incoming optical signal without 
limitation to the state of polarization of the incoming optical signal. 

34. (Withdrawn) The method of receiving an incoming optical signal of claim 
33 wherein: 

the state of polarization of the optical signal relative the local oscillator light at the 
first and second of the selected outputs is not orthogonal, and the state of polarization 
of the optical signal relative the local oscillator light at the third and fourth of the selected 
outputs is not orthogonal, and the state of polarization at the first and second outputs is 
close to orthogonal to the state of polarization at the third and fourth outputs; and 

the phase relationship between the optical signal and the local oscillator light at 
the first output is approximately 90 degrees compared to the phase relationship 
between the optical signal and the local oscillator light at the second output, and the 
phase relationship between the optical signal and the local oscillator light at the third 
output is approximately 90 degrees compared to the phase relationship between the 
optical signal and the local oscillator light at the fourth output. 

35. (Withdrawn) The method of receiving an incoming optical signal of claim 
33 wherein the step of performing computations by the digital signal processor includes 
obtaining a Jones vector representation or equivalent representation that includes the 
amplitude, phase and polarization of the incoming optical signal. 

36. (Withdrawn) The method of receiving an incoming optical signal of claim 
33 wherein the step of performing computations by the digital signal processor includes 
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producing an output which is the result of a signal processing operation on a plurality of 
samples over time of the Jones vector of the incoming optical signal. 

37. (Withdrawn) A method of receiving an incoming optical signal in a 
coherent optical detection system, said method comprising the steps of: 

emitting light from first and second local oscillators, each local oscillator not 
requiring a phase lock with the incoming optical signal; 

generating four replicas of the incoming signal and two replicas from each of the 
local oscillators light by an optical mixing hybrid; 

combining, by the optical mixing hybrid into at least four outputs; 

wherein a first output of the four outputs has a first replica of the optical signal 
and a first replica of light from the first local oscillator and the state of polarization of the 
incoming optical signal and the light emitted from the first local oscillator in the first input 
has a defined relationship; 

wherein a second output of the four outputs includes a second replica of the 
optical signal and a first replica of light from the second local oscillator and the state of 
polarization of the light from the second local oscillator with respect to the optical signal 
in the second input is close to orthogonal compared to the state of polarization of the 
light from the first local oscillator with respect to the optical signal in the first output; 

receiving optical signals from the two outputs by at least two photodetectors in 
communication with the optical mixing hybrid; 

digitizing electrical signals, by at least two A/D converters, from the at least two 
photodetectors; and 

performing computations, by a digital signal processor, on digital values from the 
at least two A/D converters to obtain information carried by the incoming optical signal. 

38. (Withdrawn) A method of receiving an incoming optical signal containing 
information in a coherent optical detection system, said method comprising the steps of: 

emitting light from a local oscillator; 

combining, by an optical mixing hybrid, the incoming optical signal and the local 
oscillator light into at least one output; 
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receiving optical signals from the output by a photodetector in communication 
with the optical mixing hybrid; and 

converting the optical signal into an electrical signal by the photodetector; 
digitizing the electrical signal by an AID converter; and 

performing computations, by a digital signal processor, on digital values from the 
AID converter, the digital signal processor producing an output which is the result of a 
signal processing operation on a plurality of samples over time of the complex envelope 
of the electric field of the incoming optical signal, wherein the output has the information 
contained in the incoming optical signal. 

39. (Withdrawn) The method of receiving an incoming optical signal of claim 
38 wherein the step of performing computations by a digital signal processor includes 
reversing at least partially the effect of propagation of the signal through an optical fiber 
transmission system. 

40. (Withdrawn) The method of receiving an incoming optical signal of claim 
38 wherein the step of performing computations by a digital signal processor includes 
reversing at least partially the effect of multipath interference imposed on the incoming 
optical signal. 

41. (Withdrawn) The method of receiving an incoming optical signal of claim 
38 wherein the step of performing computations by a digital signal processor includes 
performing an optical filtering function on the complex envelope of the electric field. 

42. (Withdrawn) A method of receiving an incoming optical signal in a 
coherent optical detection system, said incoming optical signal having a plurality of 
WDM channels, said method comprising the steps of: 

emitting light from at least one local oscillator; 

combining, by an optical mixing hybrid, the incoming optical signal and the local 
oscillator light into at least one output; 
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receiving optical signals from the output by a photodetector in communication 
with the optical mixing hybrid; 

converting the optical signal into an electrical signal by the photodetector; 

digitizing the electrical signal from the photodetector by an A/D converters; and 

performing computations, by a digital signal processor, on digital values from the 
A/D converter, the digital signal processor estimating information carried on a first of the 
plurality of WDM channels that takes into account the other WDM channels and 
subtracts crosstalk imposed on the first WDM channel by at least one of the remainder 
of the plurality of WDM channels. 

43. (Withdrawn) The method of receiving an incoming optical signal of claim 
42 wherein the crosstalk is cross phase modulation imposed on the first WDM channel 
by a second WDM channel during passage through an optical fiber transmission 
system. 

44. (Withdrawn) The method of receiving an incoming optical signal of claim 
42 wherein the crosstalk is caused by four wave mixing occurring when the plurality of 
WDM channels generates a four wave mixing product which at least partially overlaps 
the optical spectrum of the WDM channel that experiences the crosstalk. 

45. (Withdrawn) A heterodyne coherent optical detection system receiving an 
incoming optical signal in a fiber optics network, said system comprising: 

a local oscillator emitting light; 

a polarization diversity hybrid for generating two replicas of the incoming optical 
signal and two replicas of the light from the local oscillator, said polarization diversity 
hybrid combining the incoming optical signal and the local oscillator light into at least 
two outputs, wherein the state of polarization of the incoming optical signal relative to 
the light from the local oscillator at the first output is close to orthogonal to the state of 
polarization of the incoming optical signal relative to the light from the local oscillator at 
the second output; 
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wherein the incoming optical signal includes two optical channels being 
combined so that the first optical channel of the two optical channels has a state of 
polarization close to orthogonal to the second optical channel of the two optical 
channels; 

two photodetectors communicating with the optical mixing hybrid, wherein said 
photodetectors receive optical signals from the outputs and convert them to electrical 
signals; 

two AID converters to receive the electrical signals from said photodetectors, 
said AID converters digitizing the electrical signals; and 

a digital signal processor for performing computations on digital values from the 
AID converters to obtain information carried by each of the two polarization multiplexed 
channels in the incoming optical signal independent of the other channel. 

46. (Canceled) 

47. (Currently Amended) The coherent optical detection system of c l a i m A 
claim 1 wherein the digital signal processor at least partially reverses the effect of 
propagation of the incoming optical signal through an optical fiber transmission system. 

48. (Previously Presented) The coherent optical detection system of claim 
47 wherein the digital signal processor compensates for the chromatic dispersion of the 
optical fiber transmission system. 

49. (Canceled) 

50. (Canceled) 

51. (Canceled) 

52. (Currently Amended) The coherent optical detection system of claim 1 
c l a i m A whoro i n tho s i gna l process i ng operat i on performed by tho d i g i ta l s i gna l 
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proc e ssor i nc l ud e s p e rform i ng an opt i cal fi l ter i ng funct i on on th e comp le x e nv el op e of 
tho o l octr i c f i o l d. wherein the digital signal processor performs an optical filtering 
function on the electric field envelope. 



53. (Currently Amended) The coherent optical detection system of claim 1 
c l a i m 4 whoro i n tho s i gna l process i ng operat i on performed by tho d i g i ta l s i gna l 
processor i mproves tho qua li ty of tho incom i ng opt i ca l s i gna l , tho d i g i ta l s i gna l 
proc e ssor app l y i ng an a l gor i thm wh i ch ut ili z e s param e t e rs that ar e adjust e d to g i v e 
d i ff e r e nt s i gna l proc e ss i ng funct i ons, and th e va l u e s of thos e param e t e rs ar e chos e n for 
i mprov i ng tho gua li tv of tho rocovorod s i gnal, wherein the digital signal processor 
improves the guality of the incoming optical signal, the digital signal processor applying 
an algorithm which utilizes parameters that are adjusted to give different signal 
processing functions, and the values of those parameters are chosen for improving the 
gualitv of the recovered signal. 



54. (Canceled) 



55. (Canceled) 



56. (Currently Amended) The method of receiving an incoming optical signal of 
c l a i m 31 claim 30 further comprising the step of producing an output from the digital 
signal processor which is the result of a signal processing operation on a plurality of 
samples over time of the complex envelope of the electric field of the incoming optical 
signal, wherein the output has the information contained in the incoming optical signal. 



57. (Canceled) 



58. (Canceled) 



59. (Canceled) 
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60. (Canceled) 

61. (Currently Amended) The method of receiving an incoming optical signal of 
c l a i m 31 claim 30 wherein the incoming optical signal contains a plurality of WDM 
channels, further comprising the steps of: 

compensating for crosstalk imposed on a first of the plurality of WDM channels 
by at least one of the remainder of the plurality of WDM channels; and 

estimating the information carried on the first of the plurality of WDM channels. 

62. (Previously Presented) The method of receiving an incoming optical 
signal of claim 61 wherein the crosstalk is cross phase modulation imposed on the first 
WDM channel by a second WDM channel during passage through an optical fiber 
transmission system. 

63. (Previously Presented) The method of receiving an incoming optical 
signal of claim 61 wherein the crosstalk is caused by four wave mixing occurring when 
the plurality of WDM channels generates a four wave mixing product which at least 
partially overlaps the optical spectrum of the WDM channel that experiences the 
crosstalk. 

64. (New) A coherent optical detection system receiving an incoming optical 
signal in an optical communications network, said system comprising: 

a local oscillator emitting light; 

a phase diverse hybrid for generating two replicas of the incoming signal and two 
replicas of the local oscillator light, said phase diverse hybrid combining the first replica 
of the incoming optical signal and the first replica of the local oscillator light into a first 
output and combining the second replica of the incoming optical signal and the second 
replica of the local oscillator light into a second output and wherein said local oscillator 
does not have to be phase locked to the incoming optical signal; 

wherein the phase relationship between the optical signal and the local oscillator 
light in the first output is different from 0 degrees and different from 180 degrees 
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compared to the phase relationship between the local oscillator light and the optical 
signal in the second output and the state of polarization of the optical signal relative to 
the local oscillator light in the first output is not orthogonal to the state of polarization of 
the optical signal relative to the local oscillator light in the second output; and 

two photodetectors communicating with the phase diverse hybrid, wherein said 
two photodetectors receive optical signals from the two outputs and convert them to 
electrical signals; 

wherein the electrical signals received by the two photodetectors are digitized by 
two AID converters; and 

a digital signal processor performing a computation on digital values from the A/D 
converters to provide a complex representation of the envelope of the electric field of 
the incoming optical signal or a component of the complex representation of the 
envelope of the electric field of the incoming optical signal. 

65. (New) The coherent optical detection system of claim 64 wherein the 
digital signal processor produces an output which is the result of a signal processing 
operation on a plurality of samples over time of the complex envelope of the electric 
field of the incoming optical signal. 

66. (New) The coherent optical detection system of claim 65 wherein the 
digital signal processor at least partially reverses the effect of propagation of the 
incoming optical signal through an optical fiber transmission system. 

67. (New) The coherent optical detection system of claim 66 wherein the 
digital signal processor compensates for the chromatic dispersion of the optical fiber 
transmission system. 

68. (New) The coherent optical detection system of claim 67 wherein the 
signal processing operation performed by the digital signal processor compensates for 
the chromatic dispersion experienced by the optical signal by applying to the complex 
envelope of the incoming optical signal a convolution with a specified mathematical 
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function, the mathematical function being close to the impulse response of the transfer 
function corresponding to a chromatic dispersion equal in magnitude and opposite to 
the chromatic dispersion of the optical fiber transmission system. 

69. (New) The coherent optical detection system of claim 66 wherein the 
signal processing operation performed by the digital signal processor at least partially 
reverses the effect of self phase modulation imposed on the incoming optical signal. 

70. (New) The coherent optical detection system of original claim 64 wherein the 
signal processing operation performed by the digital signal processor at least partially 
reverses the effect of multipath interference imposed on the incoming optical signal. 

71. (New) The coherent optical detection system of claim 64 wherein the 
signal processing operation performed by the digital signal processor includes 
performing an optical filtering function on the complex envelope of the electric field. 

72. (New) The coherent optical detection system of claim 64 wherein the 
signal processing operation performed by the digital signal processor improves the 
quality of the incoming optical signal, the digital signal processor applying an algorithm 
which utilizes parameters that are adjusted to give different signal processing functions, 
and the values of those parameters are chosen for improving the quality of the 
recovered signal. 

73. (New) The coherent optical detection system of claim 72 wherein the 
signal processing operation that improves the quality of the recovered signal is a 
feedforward equalization-decision feedback equalization function. 

74. (New) The coherent detection system of claim 72 wherein the signal 
processing operation that improves the quality of the recovered signal is a maximum 
likelihood sequence estimation function. 
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75. (New) A method of receiving an incoming optical signal in a coherent 
optical detection system, said method comprising the steps of: 

emitting light from a local oscillator, said local oscillator not requiring a phase lock 
with the incoming optical signal; 

generating two replicas of the incoming signal and two replicas of the local 
oscillator light by a phase diverse hybrid; 

combining, by the phase diverse hybrid, the first replica of the incoming optical 
signal and the first replica of the local oscillator light into a first output; 

combining the second replica of the incoming optical signal and the second 
replica of the local oscillator light into a second output, wherein the phase relationship 
between the optical signal and the local oscillator light in the first output is different from 
0 degrees and different from 180 degrees compared to the phase relationship between 
the local oscillator light and the optical signal in the second output and the state of 
polarization of the optical signal relative to the local oscillator light in the first output is 
not orthogonal to the state of polarization of the optical signal relative to the local 
oscillator light in the second output; 

receiving optical signals from the two outputs by two photodetectors in 
communication with the phase diverse hybrid; and 

converting the optical signals from the two outputs into electrical signals; 

digitizing the electrical signals by two AID converters; and 

performing a computation on digital values from the A/D converters by a digital 
signal processor to provide a complex representation of the envelope of the electric field 
of the incoming optical signal or a component of the complex representation of the 
envelope of the electric field of the incoming optical signal 

76. (New) The method of receiving an optical signal of claim 75 wherein: 

the phase relationship between the optical signal and the local oscillator light in 
the first output of the phase diverse hybrid is a certain phase angle compared to the 
phase relationship between the local oscillator light and the optical signal in the second 
output of the phase diverse hybrid, said phase angle deviating from 90 degrees; and 
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the step of performing a computation on digital values from the AID converters by 
the digital signal processor includes compensating for the deviation of the phase angle 
from 90 degrees so as to produce a complex representation of the envelope of the 
electric field of the incoming optical signal or a component thereof that is substantially 
the same as if the phase angle were 90 degrees. 

77. (New) The method of receiving an incoming optical signal of claim 75 
further comprising the step of producing an output from the digital signal processor 
which is the result of a signal processing operation on a plurality of samples over time of 
the complex envelope of the electric field of the incoming optical signal, wherein the 
output has the information contained in the incoming optical signal. 

78. (New) The method of receiving an incoming optical signal of claim 77 
wherein the step of performing computations by a digital signal processor includes 
reversing at least partially the effect of propagation of the signal through an optical fiber 
transmission system. 

79. (New) The method of receiving an incoming optical signal of claim 78 
wherein the step of performing computations by a digital signal processor includes 
reversing at least partially the effect of the chromatic dispersion of the optical fiber 
transmission system on the optical signal. 

80. (New) The method of receiving an incoming optical signal of claim 77 
wherein the step of performing computations by a digital signal processor includes 
reversing at least partially the effect of multipath interference imposed on the incoming 
optical signal. 

81. (New) The method of receiving an incoming optical signal of claim 77 
wherein the step of performing computations by a digital signal processor includes 
performing an optical filtering function on the complex envelope of the electric field. 
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82. (New) The method of receiving an incoming optical signal of claim 75 
wherein the incoming optical signal contains a plurality of WDM channels, further 
comprising the steps of: 

compensating for crosstalk imposed on a first of the plurality of WDM channels 
by at least one of the remainder of the plurality of WDM channels; and 

estimating the information carried on the first of the plurality of WDM channels. 

83. (New) The method of receiving an incoming optical signal of claim 82 
wherein the crosstalk is cross phase modulation imposed on the first WDM channel by a 
second WDM channel during passage through an optical fiber transmission system. 

84. (New) The method of receiving an incoming optical signal of claim 82 
wherein the crosstalk is caused by four wave mixing occurring when the plurality of 
WDM channels generates a four wave mixing product which at least partially overlaps 
the optical spectrum of the WDM channel that experiences the crosstalk. 

85. (New) The method of receiving an optical signal of claim 75 wherein: 

the two digitized signals from the two A/D converters arriving at the digital signal 
processor at the same time correspond to samples taken at different times with respect 
to the envelope of the electric of the incoming optical signal; and 

the step of performing a computation on digital values from the A/D converters by 
the digital signal processor includes compensating for the different path delays 
experienced by two signal paths 

86. (New) The method of receiving an incoming optical signal of claim 79 
wherein the step of performing computations by a digital signal processor includes 
calculating the Fourier transform of an interval of time of the electric field of the optical 
signal, multiplying the Fourier transform by a set of coefficients equal to the inverse of 
the transfer function of the chromatic dispersion experienced by the incoming optical 
signal to produce a compensated Fourier transform, and calculating the inverse Fourier 
transform of the compensated Fourier transform. 
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87. (New) The coherent optical detection system of claim 67 wherein the 
signal processing operation performed by the digital signal processor to compensate for 
the chromatic dispersion experienced by the optical signal includes the calculation of 
the Fourier transform of an interval of time of the electric field of the optical signal. 
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